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Abstract 

We study the dependence of the spectral density of the covariance ma- 
trix ensemble on the power spectrum of the underlying multivariate signal. 
The white noise signal leads to the celebrated Marchenko-Pastur formula. 
We demonstrate results for some colored noise signals. 

1. Introduction 

The covariance matrix is a fundamental object in the multivariate statistics and 
probability theory. A sample covariance matrix use only part of the data and is 
determined by the number of samples. But it has the same population size as 
the covariance matrix. When the population size is not large and the number 
of sampling points is sufficient the sample covariance matrix is a good approxi- 
mate of the covariance matrix. Unfortunately, we usually investigate data with 
a sampling rate that is not sufficient and select the number of samples to be 
comparable with the population size. In this case the sample covariance matrix 
is no longer a good approximation to the covariance matrix. 

Marchenko and Pastur ^ were discussing a limiting case when the ratio p 
between the population size m and the number of samples n remains constant 
and n grows without bounds. They studied the sample covariance matrix c 
defined by the formula 

n 

Cij = x],x{ or c = xx'^ (1) 

k=l 

where stands for the normalized (i. e. with zero-mean) independent and 
identically distributed random data. The upper and lower indices denote the 
population and sample index respectively. 

The spectral density of c depends in the limit only on the variance cr^ of x 
and on the population-to-sample ratio p 
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^.VW^m^) a<X<b 
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A < a V A > 6 



where a = (l — ^/pY, b = (l + ^Jpf' ■ For p > 1, there is an additional 

Dirac measure at A = of mass 1 — -. 

P 

The formula rt2]) describes the spectral density of the sample covariance ma- 
trices of a white noise signal. So the power spectrum of the signal vector x\ is 
constant. In many situations however the signal is not accessible directly. What 
is actually measured is its filtered image. For instance if we deal with the EEG 
signal we do not measure directly the cerebral signal but only its image filtered 
through the tissues in the skull. The natural question is of course to what de- 
gree the spectral density of the sample covariance matrix depends on such signal 
filtering. We show that spectral density ^ is universal in certain circumstances 
and that it represents a special case of the general probability distribution which 
depends on the power spectrum of the signal. 



2. Signal frequency analysis and the covariance 
matrix spectral density 

The measuring device has a finite sampling rate that leads to a discrete set of the 
measured values. For that reason we will use a discrete Fourier transformation 
(DFT) for the frequency analysis. In our notation DFT is defined as 
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fn ^ 



XJe~*'^'=*^ u^ = {k- 1) 



where / is the sampling rate. 

The Fourier transform X of a real vector x is complex and fulfills 



(3) 



Xi = Xn-i, I <i <n. 



For real x it is therefore useful to use another transformation 

Re(X,) l<i< 



Xi = V2 



Im {Xn- 



+ 1 < i < n 



(4) 



(5) 



where [a] means the integer part of a. The remaining two elements are defined 
separately. Since Xi is real and equal to the sum of Xi, we define Xi = Xi. 
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For even n we take X„ = X„ since X„ is real. For odd n we use 
= y2Re(^X^^. 

The transformed vector X is real and contains the full information on the 
frequency properties of the original vector x. The definition of the covariance 
matrix ([T]) can be easily rewritten using the discrete Fourier transformation and 
the transformation (Isl): 



Q,, = xi^xi = XlXi = J2 XlXi or c = xx^ = XX* = XX^, (6) 

k=l k=l k=l 

where the rows of the matrices X and X are the transformed rows of the matrix 

X. 

Colored noise is a random signal with a non-flat power spectrum. We are 
interested in the question how the profile of the power spectrum influence the 
spectral density of the sample covariance matrix. In what follows we assume 
that the data matrix x has independent rows with identical power spectra and 
zero mean. Then the elements of the matrix X are also of mean zero - see the 
definition ([6]). Moreover the elements in the rows of the matrix x are independent. 
The transform X leads therefore also to a matrix with independent rows. Since 
the signal phase is random we get 



Re(X^)^) = (lm(X^)^) (7) 

and hence 



-^k) ) - \ { ^n-k 



+ 1 <k <n, l<i<m, (8) 



where the angle brackets denote the sample mean. To find the spectral density 
of the covariance matrix ensemble we use now the a theorem of Girko . 

Theorem 1: Let Abe a m x [cm] random matrix with independent entries of a 
zero-mean that satisfy the condition 

mVar(A,j) < B, (9) 

for some bound B < oo. Moreover, let for each m be a function Vm'- [0, 1] x 
[0, c] ^ M defined by: 

UmifJ', = myai{Aij), — < fi< , — < z/ < (10) 

m mm m 
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and suppose that Vm converges uniformly to a limiting bounded function v for 
m ^ oo. Then the limiting eigenvalue distribution p(A) of the covariation matrix 
AA^ exists and for every r > satisfies: 



p(A)dA 



1 + rA 
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M(/i,r)d/i, (11^ 



with u{fi, t) solving the equation 



r) = ^ , • (12) 



1 + r/ 



° 1 + Tju{^,r)v{^,u)d^ 



The solution of the equation ( 12 ) exists and is unique in the class of functions 



u{h,t) > 0, analytical on r and continuous on p G [0, 1]. 

Let us use this theorem taking A = X. We immediately see that c = — . 
Since all the rows of the matrix X have identical power spectra, the function 



t>(/i, u) will not depend on fi. The equation (12) shows that the function u{^,t) 
is also /i independent. Inserting v{u) and u{t) into the equations (11) and ( |I2| , 
we get 

- u{r) (13) 



1 + rA 







and 



u{r) = , \ . . . (14) 

^ 1 + tu{t)v[u) 

The spectral density is determined by the function v(j/) (that itself is a func- 



tion of the power spectrum). However, to solve the equations ([13j) and (14) for a 
general power spectrum profile is extremely difficult. So in next chapters we will 
try to get an exact formula for the spectral density at least in the simplest cases. 

3. Generalized white noise 

Consider a situation when the signals from several sources come into one 
given point. Every sources produce a noise in a specific frequency bands and the 
frequency bands are disjoint. Further, the intensity of all sources is the same. 
The total incoming signal has gaps in the power spectrum. The function v{i>) 
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Figure 1: An example of the function f (z/) for the generahzed white noise. 



(see the definition in the Theorem 1.) is a step function with steps of an equal 
hight, see the figure ([T]). 

In order to evaluate the spectral density we have to know the size d of the 
support of v{'u) (i.e. the sum of the lengths of all intervals where u(z/) is nonzero) 
and the value v of the function v{v) on this support (the function v{v) is constant 
on the support). The solution of the equation (14) gives 

- [1 + VT{d - 1)] + J[l+VT{d- l)f + AVT 



u{t) 



2VT 



and the integral equation (13) can be transformed into the form 



p(A)dA 



u 



z/ + A z/ 
The generalized Stieltjes transform 

oo 

F(z/)dz/ 



has an inverse [7j 



larg^l < TT, 



G'{yw)dw 



[l + w 



(15) 



(16) 



(17) 
(18) 

where g > and C is a contour starting at the point w = —1 and encircling the 
origin in the counterclockwise sense. For q = 1 the eq. (18) can be explicitly 
evaluated: 
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F{i^) = lim — {G{-i^ - te) - G{-u + ie)) 



(19) 



for u > 0. 

We find the spectral density p(A) as an inverse Stieltjes transform with q = 1. 



The limit in (19) with G = u and F = p leads to 

r 1 ^(Ai-A)(A-A2) Ai < A < A2 
P(A)= , (20) 

[ A < Ai V A > A2 

where A,,, = . (l T VS)^ For d < 1 . there i. an additional Dirac nrea^nre at 

A = of mass 1 — d. 

The above formula is exactly equal to the Marchenko-Pastur result (|2]). The 
existence of the Dirac measure is consequence of singularity of covariance matrices 
for d <l. 




Figure 2: Spectral density. Numerical results (stars) are compared 
with the theoretical results for p(A) with the parameters v = 1 and 
d = ?>. 



The interesting point is that while the Marchenko-Pastur result was derived 
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for a white noise signal (i.e. the power spectrum was constant over the whole 
frequency range) we get the same result also when the power spectrum has a finite 
support and contains a finite number of gaps. Moreover - the exact position of 
the gaps is irrelevant and the result depends on the total support size only. 



4. Colored noise 



Let us now pass to the case when the highs of the power spectrum segments 
are unequal. This is a quite general case since in fact any power spectrum profile 
can be approximated by a step function. 



Inserting the function f (z/) into the integral (14) and using the definition (16) 
gives 

ru{r) + f2d^-l = f2 ^/\ V (21) 

where K is number of the nonzero segments in the function f (z/). In what follows 
the symbols v and d denote vectors (in contrast to their previous meaning as 
constants) with elements Vi and di denoting the highs and lengths of segments 
respectively. 



To solve the equation (21) means to find the roots of a polynomial of degree 



{K + 1). This cannot be done explicitly. However - in similarity to the previous 



case with the bars of equal hight - the solution of the equation (21) does not 
depend on the exact location of power spectrum bars and is positive on the 
positive real axis. 

As an illustration we give the formula for the case with two steps: 

A- = 2..= (l.l),.= (i.i). (22) 
The spectral density is than 



P(A) 



- \/-27 A - 8 A3 + 36 A2 + 27 + 3 V3\/AV54 - 16 A3 + 72 - 81 A + 2 A 31 



\ 87r2A \/-27 A - 8 A3 + 36 A2 + 27 + 3 \/3%/A\/54 - 16 A3 + 72 A2 ^^^sTA 



(23) 



The spectral density of the covariance matrices will in this case again not 
depend on the exact location of power spectrum steps. Also the order of the 
steps is not important. Moreover segments of the same hight can be linked into 
one segment with the width equal to the sum of the widths of the two particulars 
segments. In this sense the spectral density does not depend on the reshuffling 
of the power spectrum. 
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1.4 
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Figure 3: Spectral density. The numerical results (stars) are compared 
with the theoretical prediction of p(A) for K = 2, v = ^1,^^, 

d= (3^,15). 



5. Summary 

The spectral density of the covariance matrix is used in many fields of physics 
and economy (see [1], [2], [3]). To analyze the system the power spectrum of the 
signal has to be taken into account. An example is the spectral analysis of the 
EEG signal 

The power spectrum directly influence the signal correlation properties. For 
instance the particular matrix elements of the covariance matrix depend on it. 
Nevertheless the spectral density of the covariance matrix ensemble remains 
nearly invariant. 

In the presented paper we discuss the spectral density of the covariance matrix 
and its dependence on the power spectrum profile of the underlying signal. The 
results show that the spectral density is invariant under the reshuffling of its 
power spectrum coefficients and hence independent on the exact spectral profile 
of the signal. 
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